Evolution of Cooperation by phenotypic similarity 
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The emergence of cooperation in populations of selfish individuals is a fascinating topic that 
has inspired much work in theoretical biology. Here we study the evolution of cooperation in a 
model where individuals are characterized by phenotypic properties that are visible to others. The 
population is well-mixed in the sense that everyone is equally likely to interact with everyone else, 
but the behavioral strategies can depend on distance in phenotype space. We study the interaction 
of cooperators and defectors. In our model, cooperators cooperate with those who are similar 
and defect otherwise. Defectors always defect. Individuals mutate to nearby phenotypes, which 
generates a random walk of the population in phenotype space. Our analysis bring together ideas 
from coalescence theory and evolutionary game dynamics. We obtain a precise condition for natural 
selection to favor cooperators over defectors. Cooperation is favored when the phenotypic mutation 
rate is large and the strategy mutation rate is small. In the optimal case for cooperators, in a 
one-dimensional phenotype space and for large population size, the critical benefit-to-cost ratio is 
given by6/c=l + 2/ v3. We also derive the fundamental condition for any two-strategy symmetric 
game and consider high-dimensional phenotype spaces. 

I. INTRODUCTION 

Evolutionary game theory is the study of frequency-dependent selection [TJ[51[31[31[5][51[7][8]. Fitness values 
depend on the relative abundance, or frequency, of various strategies in the population, for example the frequency 
of cooperators and defectors. Evolutionary game theory has been applied to understand the evolution of cooperative 
interactions in viruses, bacteria, plants, animals and humans [9j [TO] [HJ [12] US] . The classical approach to evolutionary 
game dynamics assumes well-mixed populations, where every individual is equally likely to interact with every other 
individual [4 . Recent advances include the extension to populations that are structured by geography or other factors 
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The term 'greenbeard effect' was coined in sociobiology to describe the result of the following thought experiment 
|26l 127] . What evolutionary dynamics will occur if a single gene is responsible for both a phenotypic signal ('a 
green beard') and a behavioral response (for example, altruistic behavior towards individuals with like phenotypes)? 
Later, the term 'armpit effect' was introduced |28j to refer to a self-referent phenotype that is used in identifying kin 

HUGHES]. 

Both of these concepts are now seen as cases of 'tag-based cooperation', in which a generic system of phenotypic tags 
is used to indicate similarity or difference, and the evolutionary dynamics of cooperation are studied in the context of 
these tags. A first approach, based on computer simulations, assumed a well-mixed population, a continuum of tags, 
and an evolving threshold distance for cooperation |32j . More recent models use numerical and analytic methods and 
often combine tags with viscous population structure [33 34 , 35 , 36 , 37 . A general finding of these papers is that it is 
difficult to obtain cooperation in tag-based models for well- mixed populations, indicating that some spatial structure 
is needed [14]. 

Inspired by work on tag-based cooperation [35] [33J [33] [3H] and building on a previous approach |3J], we study 
evolutionary game dynamics in a model where the behavior depends on phenotypic distance [3D] ST]. As a particular 
example we explore the evolution of cooperation [42] [43] . Studies of different organisms, including humans, support 
the idea that cooperation is more likely among similar individuals [3T] [44] [45] [46]. Our model applies to situations 
where individuals tend to like those who have similar attitudes and beliefs. We introduce a novel yet natural model 
in which individuals mutate to adjacent phenotypes in a possibly multi-dimensional phenotype space. We study one 
and infinitely many dimensions in detail. We develop a theory for general evolutionary games, not just the evolution 
of cooperation. Spatial structure is not needed for cooperation to be favored in our model. Moreover, in contrast to 
previous work [39j , we develop an analytic machinery for describing heterogeneous populations in phenotype space. 

The paper is organized as follows. In Sec. [IT] we give an overview of the main results and provide a heuristic 
derivation. Then we derive the precise condition for cooperation to be favored. This condition depends on certain 
correlations in the neutral case, that is when each individual has the same fitness. These correlations are calculated 
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FIG. 1: The basic geometry of evolution in phenotype space. There are two types of individuals (red and blue), which 
can refer to arbitrary traits or different strategies in an evolutionary game. Individuals inherit the strategy of their parent 
subject to a small mutation rate u. Moreover, each individual has a phenotype. Here we consider a discrete one dimensional 
phenotype space. An individual of phenotype i produces offspring of phenotype i — 1, i or i + 1 with probabilities v, 1 — 2v 
and v, respectively. The total population (of size N) performs a random walk in phenotype space with diffusion coefficient 
v. Sometimes the cluster breaks into two or more pieces, but typically only one of them survives. If evolutionary updating 
occurs according to a Wright-Fisher process then the distribution of individuals in phenotype space has a standard deviation 
of V2Nv. For the Moran process, the standard deviation is reduced to V Nv. 



in Section III and in Section IV the condition for cooperation is derived. We relegate some details to the appendices. 
Finite population sizes are discussed in Appendix |Aj cooperation without self interaction in Appendix [5] and the 
derivation of correlations in Appendix [C] In Appendix [D] we show that all results in the large population size limit 
are identical for the W-F and for the Moran process. In Appendix [E] we consider general payoff matrices, and finally 
in Appendix [F] we discuss an infinite dimensional phenotype space. 



II. OVERVIEW OF MAIN RESULTS 



Consider a population of asexual haploid individuals, with a population size N that is constant over time. Each 
individual is characterized by a phenotype, given by an integer i that can take any value from minus to plus infinity. 
Thus, this phenotype space is a one-dimensional and unbounded lattice. Individuals inherit the phenotype of their 
parent subject to some small variation. If the parent's phenotype is i, then the offspring has phenotype i — 1, i or 
i + 1 with probabilities v , 1 — 2v and v, respectively. The parameter v can vary between and 1/2. 

Let us consider a Wright-Fisher process. In each generation, all individuals produce the same large number of 
offspring. The next generation of individuals is sampled from this pool of offspring. To introduce some fundamental 
concepts and quantities, we first study the model without any selection. No evolutionary game is yet being played, 
and there is only neutral drift in phenotype space. The entire population performs a random walk with a diffusion 
coefficient v, and by this process will tend to disperse over the lattice. In opposition to this, all of the individuals in 
the population will be, to some degree, related due to reproduction in a finite population. Thus, while occasionally 
the population may break up into two or more clusters, typically there is only a single cluster |47[ 148] . The standard 
deviation of the distribution in phenotype space, which is a measure for the width of the cluster, is y/2Nv. 

Next, we superimpose the neutral drift of two types: the strategies A and B (Fig. [TJ . Still for the moment assuming 
no fitness differences, we have reproduction subject to mutation between A and B. Specifically, with probability u the 
offspring adopts a random strategy. The mutation-reproduction process defines a stationary distribution [49] ■ If u is 
very small relative to N, the population tends to be either all- A or all-B. If u is large, the population tends toward 
one half A and one half B. Figure [2] illustrates the random walk in phenotype space of the population comprised of 
the two types A and B. 

Using coalescence theory |50L I51j many interesting and relevant properties of the distributions of both the strategies 
and phenotypic tags can be calculated. For example, the probability that two randomly chosen individuals have the 
same phenotype is z — 1 /\/8Nv. The probability that two randomly chosen individuals have the same strategy and 
the same phenotype is g — z{\ — Nu/2). The probability that two individuals have the same strategy and a third 
individual has the same phenotype as the second is h = z[l — Nu(2 + \/3)/4]. These results hold for large population 
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FIG. 2: Random walks in phenotype space. The figure shows two computer simulations of a Wright-Fisher process in a one- 
dimensional discrete phenotype space. The phenotypic mutation rate is v = 0.25. The colors, red and blue, refer to arbitrary 
traits, because no game is yet being played. All individuals have the same fitness. The population size is (a) N — 10 and 
(b) N = 100. The strategy mutation probability (between red and blue) is u = 0.004. Therefore, a given color dominates on 
average for 2/u — 500 generations (since new mutations arrive at rate Nu/2 and fixate with probability 1/N). The standard 
deviation of the distribution in phenotype space is y2Nv. About 95% of all individuals are within 4 standard deviations. 
Often the population fragments into two or several pieces, but only one branch survives in the long run. We use the statistics 
of these neutral 'phenotypic space walks' for calculating the fundamental conditions of evolutionary games in the limit of weak 
selection. 



size N and small mutation rate u; more precisely, we assume large Nv and small Nu. The relevance of z, g and h 
will become clear below. The expressions for z, g and h are derived for general Nv and Nu in Section [III| where they 
appear as eqs (121, (21 1 and (26), respectively. 

We can now use these insights to study game dynamics. We investigate the competition of cooperators, C, and 
defectors, D. Cooperators play a conditional strategy: they cooperate with all individuals who are close enough in 
phenotype space and defect otherwise. The notion of being close enough is modeled by a lattice structure. In particular, 
a cooperator with phenotype i cooperates only with other individuals of phenotype %. Defectors, in contrast, play an 
unconditional strategy: they always defect. Cooperation means paying a cost, c, for the other individual to receive a 
benefit b. The larger the total payoff of an individual, interacting equally with every member of the population, the 
larger the number of offspring it will produce on average. We want to calculate the critical bencfit-to-cost ratio, b/c, 
that allows the game in phenotype space to favor the evolution of cooperation. 

A configuration of the population is specified by mj and n i; which are the number of cooperators with phenotype 
i and the total number of individuals with phenotype i, respectively. The total payoff of all cooperators is Fq — 
^2 i rrii(bmi — cni). The total payoff of all defectors is Fjj — — Wj)&mj. There are X)i m * cooperators and 

N — J2i m i defectors. The average payoff for a cooperator is fc = Fc/^2i m i- The average payoff for a defector is 
fu = Fjj /(N—J2i m i)- Cooperators have a higher fitness than defectors if fc > Id, which leads to m^brrii — crii) > 
J2i m i J2j m j n jQ > ~ C )/N ■ Averaging these quantities over every possible configuration of the population, weighted 
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FIG. 3: Excellent agreement between numerical simulations and analytic calculations. We show the critical benefit-to-cost ratio 
that is needed for cooperators to be more abundant than defectors in the stationary distribution. We have used a Wright-Fisher 
process with a phenotypic mutation rate v — 1/2 and a strategy mutation probability u — 1/(2N). The red line indicates the 
result of our analytic calculation. For these parameter values the asymptotic limit for large N is b/c = (1 + 12v2)/7 ~ 2.5672. 
The red dots indicate the result of numerical simulations. The grey line illustrates the critical 6/c-ratio for u — > with the 
asymptotic limit b/c = 1 + 2/V3 » 2.1547. 



by their stationary probability under neutrality, we obtain the fundamental condition 

i i ij 

Under this condition cooperators are more abundant than defectors in the mutation-selection process. The above 
argument and our results are valid in the weak selection limit. A precise derivation of this inequality is presented in 
Section |IV| Correlation terms similar to the ones above sometimes arise in studies of social behavior and population 
dynamics [261152"] . The first two terms in inequality (JlJ are pairwise correlations, while the third is notably a triplet 
correlation. Note that the argument leading to inequality ([ij includes self-interaction, but that the effect of this 
becomes negligible when N is large. 

When the population size is large, the averages in inequality ([TJ are proportional to the probabilities g, z and h 
respectively, which we introduced earlier. Consequently, inequality ([IJ can be written as bg — cz > (b — c)h. Using 
the values of z,g, h given above we obtain 

b/c>l + -j=, (2) 

which is approximately 2.16. If the benefit-to-cost ratio exceeds this number, then cooperators are more abundant 
than defectors in the mutation-selection process. The success of cooperators results from the balance of movement 
and clustering in phenotype space. Inequality ([2| represents the condition for cooperators to be more abundant than 
defectors in a large population when the strategy mutation rate u is small (Nu 1) and the phenotypic mutation 
rate v is large (Nv 3> 1). We derive conditions for any population size and mutation rates. Figure [3] shows the 
excellent agreement between numerical simulations and analytical calculations. In general we find that both lowering 
strategy mutations, and increasing phenotypic mutations favor cooperators. 



III. CORRELATIONS IN THE NEUTRAL CASE 



Let us give a more precise definition of the model first. Consider a population of N haploid individuals (players). 
Each individual k = l,...,N has an integer- valued phenotype Xk G Z, which we also refer to as its position in 
phenotype space. Additionally, each individual has a strategy Sk € {0, 1}, and we refer to these two strategies as 
cooperation (1) and defection (0). In general, players' phenotypes and strategies determine their fitness. In the 
Wright-Fisher (W-F) process, each of the N individuals of the next generation independently chooses a parent from 
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the previous generation with a probability proportional to the parent's fitness. Each offspring inherits the parent's 
position (phenotype) with probability 1 — 2v, and it is placed to either the left or the right neighboring position of 
the parent, both with probability v. Each offspring also inherits the parent's strategy with probability 1 — u, and it 
adopts a random strategy with probability u. 

In this section we consider the neutral case, that is when all players have the same fitness. Note that the strategies 
and the phenotypes of the individuals change independently, and evolve according to the Wright-Fisher process 
[471 148j . The system rapidly reaches a stationary state where the individuals stay in a cluster with variance 2Nv, 
but the cluster as a whole diffuses over the space (the integers) with diffusion coefficient v. We are interested in the 
properties of this stationary state. 

We are particularly interested in four probabilities. We pick three distinct individuals k, q, and I from the population 
in the stationary state. For their phenotypes and their strategies we define the following four probabilities 



V = 


Pr(S k = 


s q ) 




z = 


Pv(X k = 


--x q ) 




9 = 


Pr(S fc = 


Sq, Xk 


= x q ) 


h = 


Pr(S* = 


St, Xk 


= x q ) 



(3) 



In words, y is the probability that two individuals have the same strategy, and z is the probability that they have 
the same phenotype. They have simultaneously the same strategy and phenotype with probability g. Out of three 
individuals, the probability that the first two have the same phenotype, and simultaneously the first and the third 
have the same strategy is denoted by h. Note that neither g nor h factorizes in general. 

To obtain the above probabilities we have to know the probability Pr(T = t) that the time T to the most recent 
common ancestor (MRCA) of two randomly chosen individual is T — t. This time is not affected by either the 
strategies or the phenotypes of the players. It is determined solely by the W-F dynamics. The ancestry of two 
individuals coalesce with probability l/N in each time step. Hence the probability that the time to the MRCA is t is 

P r (T = t)=(l-l) t_1 l (4) 

We can continue the calculation for finite system size N, but the expressions become cumbersome. Hence we 
relegated the finite N calculations to Appendix [Aj where we mainly treat the special v = 1/2 case. In this section we 
discuss the large population limit N — > oo, where we introduce the rescaled time r = t/N . In this limit we can use a 
continuous time description, where the coalescent time distribution Q is given by the density function 

p(r) = e" T (5) 

and the average coalescence time becomes r = 1 in the new unit. 

Due to the non-overlapping generations in the W-F model, each individual is a newborn and has the chance to 
mutate both in strategy and phenotype space. In the large N and u, v — > limit, the system can be described as a 
continuous time process. Strategy mutations arrive at rate y, = 2Nu and phenotype mutations at rate v = 2Nv (in 
each direction) on the ancestral line of two individuals. Note that this continuous time limit is exact for the Moran 
process even for finite values of v, as it is shown in Appendix [D] In the W-F model, for finite values of v we have a 
discrete time random walk, but the typical number of steps goes to infinity. In that limit the discrete and continuous 
time walks become identical, and hence the finite v behavior can be recovered as the v — > oo limit. 



A. Phenotypic distance 

Let us first study the phenotypes of the players. Here we calculate not only z, but in general the probability that 
two randomly chosen individuals k and q are at distance x in phenotype space 

z(x) = Pr(X k -X q =x) (6) 

We know that the (signed) distance between the two individuals changes by plus or minus one at rate v, and the 
distance distribution after time r can be expressed in terms of the Modified Bessel functions [53J |M] as 



C^|r) = e- 2 ^7| x |(2i/r) 



(7) 
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The probability that two individuals are distance x apart is 

oo 

z(x) = Pl '( x k ~Xq = x\T = t)Pr(T = t) (8) 
t=i 

which becomes an integral of the corresponding density functions in the continuous time limit 



z(x) = I p(r)C(x\r)dT= I e- (2 » +1 ^I lxl (2vT)dT (9) 
o o 



By using the identity 



n 



6-t (a- Va 2 -& 2 )' 



e~ ac I^(bc) dc = v ; * ^ (10) 



we arrive at the probability distribution of the signed distance 



V4i/+ 1 V 2l/ 
The individuals are at the same position with probability 



V4^ + 1 



Distribution (111 is of course normalized X)x=-oo z i x ) = 1j anc ^ its second moment is 

OO 

x 2 z(x) = 2i/ (13) 



a; — — oo 



Note that this second moment is twice the variance of the individual positions, which is exactly v — 2Nv even for finite 
N (see Appendix [X]) . Hence the individuals stay together in a cluster of size \/2Nv. This cluster diffuses collectively 
through phenotype space. If one follows the ancestral line of an individual time t back, its position x(r) will change 
by one at rate v/2 in each direction. Consequently, the position of the cluster has a variance proportional to time 

(x 2 ) =vt= 2vt (14) 

which implies a diffusive motion. The same result is valid for any finite N in the large time limit. Note that the 
diffusion coefficient D = v does not depend on the population size. Since the cluster itself wanders in space, the 
average number of individuals at any given site goes to zero. That is why we focus on distances in the phenotype 
space 

B. Pair with same strategy 

We are interested in the probability y that two randomly chosen individuals have the same strategy. In the 
continuous time limit, strategy mutations arrive at rate \x on the ancestral lines of the two individuals. The two 
individuals have the same strategy if there were no mutations, which is the case with probability e~ MT . Otherwise 
there was at least one mutation, hence at least one of the players has a random strategy, so they have the same 
strategy with probability 1/2. Consequently, the probability that two players have the same strategy time r after 
their MRCA is 

y(r) = e-^ + \{l-e-^) (15) 
The probability y that two randomly chosen individuals have the same strategy is 

oo 

y = J2P*(Sk = S q \T = t)Pr(T = t) (16) 
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In the continuous time limit we obtain 



JJ = / p(T)y(T)dT = 



2 + m 
2(1 + M) 



(17) 



where we have used ^ and (151 



C. Pair with same strategy and phenotype 

The probability g that two randomly chosen individuals have the same phenotype and also have the same strategy 
can be obtained as 



g = Y,Pr{S k = S q \T = t)Pr(X k = X q \T = <)Pr(T = t) 



(18) 



Here we have used the property that although g does not factorize in general, nevertheless for any given time t the 
conditional probabilities factorize as 



Pr(S fe = S q , X k = X q \T = t)= Pv(S k = S q \T = t)Pr(X k = X q \T = t) 



(19) 



The reason is that mutations occur completely independently in the strategy and the phenotype space. The corre- 
sponding integral in the continuous time limit hence becomes 



9=1 p(r)y(T)C(T)d,7 



(20) 



where we use the notation £(t) = £(0|t). Note that it is also easy to obtain the analog probability where the phenotype 
difference is x, but we do not consider that here. Using identity (10 1 again, we can evaluate the above integral 



1 1 

2y/l + Av 2 v /(l + /i)(l + /i + 4^) 



(21) 



D. Three point correlations 

Now we turn to the calculation of the three point probability h which is defined in (|3| . If we follow the ancestral 
lines of three individuals back in time, the probability that there was no coalescence event during one update step is 
(1 — 1/N)(\ — 2/N). Two individuals coalesce with probability 3/N • (1 — l/N). When two individual have coalesced, 
the remaining two coalesce with probability 1/iV during each update step. Hence the probability that the first merging 
happens to any pair of individuals at time t% > 1 back in time, and the second t2 > 1 before the first one is 



Pr(«3,*2) 



3 

AT2 



1 

N 



2 

N 



1*3-1 



1 

N 



The probability that three individual coalesce simultaneously at time £3 is 



Pr(t 3 ,0) 



N 2 







(K) 









In the N — > 00 limit ( 22 1 converges to the density function 

p(T 3 ,T 2 ) = 3e-( 3T3+T2 ) 



(22) 



(23) 



(24) 



with T3 = t^/N and T2 = t2/N. Note that (23) does not affect the large N limit. 

Let us call the scaled time when individuals q, k coalesce r qk , and when k,l coalesce T k \. With probability 1/3 
individuals q, k coalesce first at r qk = t 3 and they coalesce with / at T k \ = r 3 + t 2 . Similarly with probability 1/3 
individuals k, I coalesce first at r k i = T3 and they coalesce with q at r qk = T3 + t 2 . If, however, I, q coalesce first with 
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probability 1/3, it makes r q k — t^i — t 3 + t 2 . Since we know the probability density y(r) that two individuals with 
a MRCA at time r back have the same strategy (15 1, and the probability density ((t) = C(0|t) that they are at the 
same position ([7]), we can simply obtain the three point correlation as 

00 00 

h =\j dT zj dr 2 Pfo> T 2) [Cfo)«fo + t 2 ) + C(r 3 + T 2 )y(T 3 ) + C(r 3 + T 2 )y(T 3 + r 2 )] (25) 


This integral can be evaluated by first introducing a variable for t 2 + t 3 in the last two terms of the integral, and by 
using identity (10) in all three terms. We obtain 

h = (l + M)(3 + ^) + Ci(2 + /i)-/iC 3 
2(1 + fi) (2 + fi)VTT4v 

with the shorthand notation 



l/(i + /,)(! + 4,) 

By now we have obtained all the correlations in (J3J) in the N — ► 00 limit for any values of v and \x. 



IV. THRESHOLD b/c RATIO 



In this section the individuals play a simplified Prisoner's Dilemma game given by the payoff matrix 





when playing 


; against 




C 


D 


c 


b-c 


—c 


payoff of 






D 


b 






(28) 



Here b > is the benefit gained from cooperators, and c > is the cost payed by cooperators. We assume that all 
individuals interact (in this sense the population is "well mixed"). Cooperators, however, play a conditional strategy: 
they cooperate with other individuals who have the same phenotype, and they defect otherwise. Defectors always 
defect. The total payoff of an individual is the sum of all payoffs that individual receives. We introduce the effective 
payoff of an individual / = 1 + S ■ payoff, where S > is the strength of the selection, and 8 = corresponds to the 
neutral case discussed in Section |TTT] Note that S must be sufficiently small to make all fitness values positive. 

We consider here the simplest possible case, where each individual also receives a payoff from self interaction. 
Excluding self-interaction results in a 1/N correction, which is discussed in Appendix [B] An extension to a general 
payoff matrix is considered in Appendix |EJ 



A. Fitness 



Let rij denote the number of players of phenotype i, and the number of cooperators of phenotype i. A state of 
the system is given by the vectors s = (n,m). Let fc,% and fo,i represent the (effective) payoffs of a cooperator and 
a defector, respectively, of phenotype i. When self interaction is included these values are 

fc.i = 1 + 5 [bm t - crii] 
fD,i = l + 5[brrii]. 

Let wc,i and W£>,i represent the fitness (i.e. average number of offsprings) of a cooperator and a defector of 
phenotype i. After one update step (which is one generation) we obtain 

Nfci 

W C ,z = ==-, 7 —, T7 T 30 
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Here a cooperator is chosen to be a parent with probability given by its payoff relative to the total payoff, and this 
happens N times independently in one update step. The denominator of (j30h can be written as 



^2[ m jfc,j + ( n j - mj)fD,j] = N + 5(b - c)^2 m j n j ( 31 ) 

3 3 

Therefore, in the S — > limit, we obtain the fitness of a phenotype i cooperator 

wc,i = 1 + 8 I brrii — crii j-j— rrijUj J + 0(<5 2 ) (32) 



B. Effect of selection 



Let p denote the frequency of cooperators in the population. Cooperation is favored if cooperators are in the 
majority at the stationary state, (p) > 1/2. The frequency of cooperators p changes during one update step due to 
selection and due to mutation. In any state s of the system, the total change of cooperator frequency can be expressed 
in terms of the change due to selection as 

Ap to t(s) = (1 - u)Ap scl (s) + u Q - p^j (33) 

Here the first term describes the change due to selection in the absence of mutation, which happens with probability 
X—u. The second term stands for the effect of mutation, which happens with probability u to each player independently. 
In this latter case the frequency p increases in average by 1/2 due to the introduction of random strategies, and 
decreases by p due to the replacement of cooperators. 

In the stationary state (p) is constant, hence the total change of frequency vanishes (Ap) to t = 0. Then from (33 1 
we can express the average cooperator frequency with the change of frequency due to selection as 

(p) = \ + ^(Ap) scl (34) 

This means that by calculating the average change of cooperator frequency, we also obtain the average cooperator 
frequency. It also means that cooperators are favored (p) > 1/2 if their change due to selection is positive in the 
stationary state 

(Ap) sel > (35) 

Now let us perform a perturbative expansion for small selection S <C I. In a given state s = (n, m), the expected 
change of p due to selection in one update step is 



(36) 

This expression vanishes for S = for the fitness function (|32|. (Note that this statement is not true in general for 
arbitrary models). Its Taylor expansion is 



A K .) = + S^M _ + _ i £ ,„, ^ _ + 0(O ,37) 



d5 



5 \ ^ dwc 



a=o v ' N ^ dS 



5=0 



We also expand the stationary probabilities of finding the system in state s 

7r(s) = ir(°\ S ) + 5ir (1 \s) + 0(S 2 ) (38) 

where 7n°)(s) is the stationary probability in the neutral state (here we consider two states equivalent if they only 
differ by translation along the phenotype space). Consequently, in the stationary state in the presence of the game, 
the average change in cooperator frequency can be expressed in the leading order in terms of averages in the neutral 
stationary state 

<Ap>sei=^(£m^\ +0(J 2 ) (39) 
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This expression has to be positive for cooperation to be favored (35). Here the subscript refers to S = 0, that is to 
an average taken in the stationary state of the neutral model (-)o = Es ' ( s )- More generally, one can also easily 
obtain higher order terms in 5 based on ( 37 I and ( 38 1 . The first derivative of the effect of selection in the stationary 
state 



(Ap) U) ^ d(A P ) scl 



dS 



8=0 



can be obtained from ( |39[ ), by using the fitness (|32j of our model, as 



(Ap) 



Ci) 



N 



lib; ) — C < 



E 



mill; 



b-c 
N 



(40) 



(41) 



The threshold model parameters are then obtained when the change (Ap)£j 
(|35l 



(Ei«W)o N 



HE 



i. j 1 J 3 



'0 N 



HE 



0, as follows from the general condition 



(42) 



Hence, we have expressed the threshold b/c ratio in the small selection limit in terms of correlations in the neutral 
stationary state. Note that the averages in (41 1 cannot be moved inside the sum, since at any given position any 



stationary average is zero. Also note that all terms in (42) are of order N 



The above derivation is valid for finite N and S — * 0. We are also interested, however, in the N — * oo asymptotic 
behavior. In that case all the above derivation can be repeated when simultaneously 6N — » 0. 

Expression (41 1 for the change in cooperator frequency can be rewritten in a more intuitive way. First we express 



the total payoffs of cooperators and defectors respectively as 

fc = J2 m ^ c ' i=Nc+SFc 

i 

fD = J2 m ^D, l ^N D + 5F D 

i 

in a given state, where Fq and Fd are the total payoffs without considering weak selection 

Fg = ^ m,i(bmi - cm) , F D = J^(nj - m^brrii 



(43) 



(44) 



and Nc — Ei m * anc ^ = N — Nq are the number of cooperators and defectors respectively. With this notation 
the change in cooperator frequency (p7|) can be rewritten as 



Ap(«) = ^ (N D F C - N C F D ) + 0(<5 2 ) 



(45) 



This expression was obtained in an intuitive way in Section [TTJ By averaging over the stationary state we of course 
recover (41 1. 



C. Threshold value from correlations 



Let us now evaluate the expected values in (42 1. We randomly choose three individuals k, q, and / with replacement. 



All expected values in ( 42 1 can be expressed in terms of probabilities in the neutral stationary state 

( XX ) = N 2 p r(Sfe = S g = 1, X k = X q ) 



E 



rmm ) = N 2 Pr(5 fc = 1, X k = X q ) 



E 



mimjUj ) 
' o 



= Ar 3 Pr(S ; = S fc = 1, X k =X q ) 



(46a) 
(46b) 
(46c) 
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The indices i and j refer to positions, while k,q and I refer to individuals. These identities are self explanatory, 
nevertheless they are proven in Appendix O 



Because the two strategies are equivalent in the neutral stationary state, all expressions (46) remain valid when we 
change any 1 to 0. Consequently all expressions (46 1 simplify to 




N 2 
N 3 



Pr(S' fc — S q , X k — X q ) 

Pr(A fe = X q ) 

Pt(Si — Sk, X k = X q ) 



(47) 



Note that these probabilities are denoted in Section |TT] as P%, Pi, and P3 respectively. Substituting the probabilities 
of ( 47 ) into (42 I we arrive at the general condition expressed in terms of two and three point correlations 



Pv(Si = Sk, X k = X q ) - Fr(X k = X q ) 
Pr{Si = S k , X k = X q ) - Pr(5 fc = S g , X k = X q ) 



(48) 



In Section III we have calculated similar probabilities defined in Q, but always for two different individuals. In 
other words while in the probabilities of (47 1 we pick two individuals with replacement, in the quantities of ^ 
two individuals were picked without replacement. We know, however, that out of two individuals we pick the same 
individual twice with probability l/N, and pick two different individuals otherwise. We also know the corresponding 
probabilities when picking three individuals. With this knowledge we can express the probabilities with replacement 
in (47) with the probabilities without replacement in ^ as follows 



Pr(S fe 



Pr(S, 



Sq, X k 



Pr(X k 

Sk, x k 



x„ 



x„ 



x„ 



1 

N 
1 

N 
1 



[{N- 1)9 + 1] 
l(N-l)z + l] 

[(N - 1)(N -2)h+(N-l){z + y + g) + 1] 



(49) 



Now we substitute these probabilities into condition ( 48 1 to obtain the threshold condition 

(N - 2)(z -h) + l-y + z-g 



(N-2)(g-h) + l-y-z + g 



(50) 



The above condition (50) is exact for any finite N with self interaction. Without self interaction a 0(l/iV) correction 
appears as discussed in Appendix [B] The model of course makes no sense for N = 1, and the smallest interesting 
population size is N = 2. In the N — > 00 limit of (50 1 we also obtain a simple rule 



(51) 



Substituting the expressions (12 1, (21 1, and (26 1 into the above equation for z, g, and h respectively, we arrive at 



M C 3 ~ (2 + + (1 + n) 2 
M C 3 + (2 + M )C 1 -(l + /x) 



(52) 



where we have used the shorthand notation (27). This is our main result: the exact threshold b/c ratio in the N — > 00 
and weak selection limit. For parameter values b/c > (b/c)* there are more cooperators than defectors in the system 
in the long time average. 

In Figure|4] we plot the exact (b/c)* ratio (52 1 as a function of [i for several values of v. One observes that (b/c)* 
gets smaller both for smaller /1 and for larger v . Hence small strategy mutation and large phenotype mutation helps 
cooperation. The large v limit includes the finite v (phenotype changing probability) case. Note that since the cluster 
size in phenotype space is \/2Nv, the average number of individuals with the same phenotype is proportional to 
yjN/v, hence there are plenty of individuals to interact with even for finite v values in the large iV limit. 
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FIG. 4: Exact threshold b/c ratio ( |52| | in the N — > oo limit for several values of Cooperation is most favored in the /i — > 
and i/ — > oo limit, where (b/c)* = 1 + 2/v3. 



In the v — > oo limit ( 52 1 becomes 

'b 



= (2 + M )yr+7I-2(l + Af ) 2 - M V3H^ 1, 
cy -(2 + /x)vT+7t + 2(l + A) -mV3Tm v^' 



which for — > behaves as 



b\* , 2^3 7^-3 a , 



which is w 2.16 in the leading order. For /i — > oo the threshold ratio (53) diverges as 



-) =^+l + 0(^) 



Conversely, in the ijl 



limit ( 52 1 becomes 

_ \/3(l + Avfl' 2 + (3 + 8j/)V3 + 4j/ 
V3(l + 4i/) 3 / 2 - VsTAv 



•0( M ) 



(53) 



(54) 



(55) 



(56) 



This limit function diverges as 3/4i/ for small i/, but converges to the constant 1 + 2/y3 as ^ — > oo. Hence the best 
scenario for cooperation is /i — > and f — > oo where (b/c)* = 1 + 2/y/3. 

The large AT asymptotic results are identical for the Moran process, where we choose a random individual to die, 
and another (with replacement) to reproduce with probability proportional to the player's payoff (see Appendix [D]) . 

We would like to briefly comment on the relationship between our work and inclusive fitness or kin selection theory 
[2"B1, [3~6 58 . Let R be the inverse of the r.h.s. of (48 1. Now we formally obtained Hamilton's rule (b/c)* = l/R. By 
dividing both the numerator and the denominator in R by Pr(X k = X q ), (we can assume that it is not zero), and 
using the definition of conditional probability, we can rewrite R as 



R 



Pr(S fc 



Sq\ X k 



X q )-Pi(Si = S k \ x k =x q ) 



l-Pr(Si = S k \ X k = X q ) 



(57) 



Now with the notation 



G = Pr(5 fc 



Sg\ X k 



X q ), G = Pr(S l = S k \X k =X q ) 



(58) 
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we obtain R — (G — G)/(l — G), which is in the form of usual relatedness formula. Note, however, that this G is 
not the probability of identity in state (IIS) between two random individuals in the population as it usually is in 
inclusive fitness theory. Instead, G is a sort of weighted average of IIS probabilities in which those who share the 
same phcnotype with more players are assigned a larger weight. 



V. CONCLUSIONS 



We have derived the conditions for cooperation to be favored for games in phenotype space for any population size 
and mutation rate. Figure [3] shows the excellent agreement between numerical simulations and analytical calculations. 
The argument that leads to inequality ([!]) contains self-interaction, which means that each cooperator adds b— c to his 
payoff. Typically, self-interaction is not a desirable assumption, but it docs simplify the calculation. Excluding self- 
interaction requires us to calculate two more correlation terms (see Appendix [B]). But in the limit of large population 
size, the difference between the two approaches results only in a 1/iV correction term for the critical benefit-to-cost 
ratio. Thus, the crucial condition Q holds for the case with and without self-interaction. 

In Appendix IE] we expanded our analysis to study any 2x2 game, not only the interaction between cooperators 



and defectors. Here the general payoff matrix is given by (El I. For the game in a one dimensional phenotype space 



and large population size we find that C is more abundant than D if 

{R-P){l + V3)>T-S. (59) 

This formula can be used for evaluating any two-strategy symmetric game in a one dimensional phenotype space. We 
have discussed the snow-drift game and the stag-hunt game as particular examples. 

We can also study higher dimensional phenotype spaces. In general, for higher dimensions it is easier for cooperators 
to overcome defectors. The intuitive reason is that in higher dimensions phenotypic identity also implies strategic 
identity. In Appendix |F| we show that in the limit of infinitely many dimensions, and under the same assumptions that 



produced conditions d2j) and (59), the crucial benefit-to-cost ratio in the Prisoner's Dilemma converges to b/c > 1. For 



general games, the equivalent result of condition (59 1 becomes R> P, which means the evolutionary process always 
chooses the strategy with the higher payoff against itself. Our basic approach can also be adapted to continuous, 
rather than discrete, phenotype spaces. In this case, no two individuals have exactly the same phenotype, but the 
conditional behavioral strategy is triggered by sufficient phenotypic similarity. 

In summary, we have developed a model for the evolution of cooperation based on phenotypic similarity. Our 
approach builds on previous ideas of tag based cooperation, but in contrast to earlier work [33l [34j |35l [36j |37j , we do 
not need spatial population dynamics to obtain an advantage for cooperators. We derive a completely analytic theory 
that provides general insights. We find that the abundance of cooperators in the mutation-selection equilibrium is an 
increasing function of the phenotypic mutation rate and a decreasing function of the strategic mutation rate. These 
observations agree with the basic intuition that higher phenotypic mutation rates reduce the interactions between 
cooperators and defectors, while higher strategic mutation rates destabilize clusters of cooperators by allowing frequent 
invasion of newly mutated defectors. Therefore, cooperation is more likely to evolve if the strategy mutation rate is 
small and if the phenotypic mutation rate is large. In a genetic model this assumption may be fulfilled if the strategy 
is encoded by one or a few genes, while the phenotype is encoded by many genes. Also in a cultural model, it can 
be the case that the phenotypic mutation rates are higher than the strategic mutation rates: for example, people 
might find it easier to modify their superficial appearance than their fundamental behaviors. Furthermore, we show 
how the correlations between strategies and phenotypes can be obtained from neutral coalescence theory under the 
assumption that selection is weak |51[ 158] . Our theory can be applied to study any evolutionary game in the context 
of conditional behavior that is based on phenotypic similarity or difference. 
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APPENDIX A: FINITE POPULATIONS FOR v = 1/2 



Here we consider the Wright-Fisher (W-F) model for finite N and v = 1/2. What makes this case simple is that 
at each time step all individuals move. The probability that the time to the MRCA is t is given by Q. During t 
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generations there are exactly 2t birth events in the ancestry of two individuals, and in the v =1/2 case the phenotypic 
distance between two individuals follows a simple random walk with two steps in phenotype space per one time unit. 
Consequently, the distance between two siblings is always even. After some transient time the whole population will 
be constrained on the same sub-lattice of even, and then odd sites. The distance distribution of two individuals k 
and q, time t after their MRCA is 



Pi(X k -X q = x\T = t) = 2" 



2t 
t + x/2 



(Al) 



where again x is always even. Consequently the probability z(x) that two randomly chosen individuals are at distance 
x apart can be obtained from (pi 



z(x) 



N 



1 CO 



2t 

t + x/2 



N - 1 
AN 



(A2) 



This sum can be evaluated using the identity 



^/ 2t 
^ [t + x/2 



to obtain 



1 



z(x) 



Vn + i 

N 



N - 1 



1*1/2 



x = 
Id > 2 



x = 
Id > 2 



(A3) 



N - 1 \N + 2y/N+ 1, 

Hence, apart from the special x = case, z(x) decays exponentially in x. For fixed distances and 



(A4) 



the 



asymptotic behavior is z{x) = 1/VA+ 0(1/N). The second moment of the distance distribution (A4| is simply 2A" 



Now we turn to the strategies of the individuals. The strategies of the two players are the same if no mutations 
happened during time t to either player, which is the case with probability (1 — u) 2t . Otherwise the two strategies 
are the same with probability 1/2. Consequently, the conditional probability is 



y { t) = (l-uft+\[l-{l-ur\= l -±f- 

where we introduce the shorthand notations 

U = (l-u) 2 , M = N(l — U) + U 
The probability y that two randomly chosen individuals have the same strategy becomes 



y 



oo 1 , 

$>(%(*) = , 1 



u 

M 



(A5) 



(A6) 



(A7) 



where we have usedffl and ( A5 1 . 

Similarly, using ( |18| we obtain the probability g that two randomly chosen individuals have both the same strategy 
and the same phenotype 



1 



2(VN+1) 



U 

2\[M (VN + VM 



(A8) 



These are exact results for arbitrary number of individuals N and mutation rate u. In the N 
of the formulas (A4|, (A7) and (A8l with [i = 2Nu kept constant, we recover the v 



oo and u — > limit 
oo limits of the corresponding 
formulas (|11|, (|17[) and (|21|, apart from a factor two. This factor two is a peculiarity of the v = 1/2 case. Since here 



the distance between individuals is always even, there must be twice as many players at a given even distance. Note 
also that the variance of the cluster is 2v both for v = 1/2 and for the continuous limit calculation. 
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For only two individuals, the general condition (50 1 simplifies to 



N=2 



1 - y- z+g 



(A9) 



which contains only quantities we have just calculated in this section. To obtain the exact (b/c)* for any other finite 
N we have to use the general expression ( 50 1 , and obtain h analogously to ( 25 1 and using ([22]) and ( 23 1 . The formulas 



for h and (b/c)* are too cumbersome to include here. We have, however, checked these formulas with computer 
simulations for many values of N. We explicitly simulated the W-F process and found the threshold (b/c)* value 
where the frequency of cooperators in the stationary state becomes larger than 1/2. Moreover, in the TV — * oo, u — > 
limit with /x = 27V it constant, we recover the continuous time formula (53). 



APPENDIX B: EXCLUDING SELF INTERACTION 



If cooperators cannot interact with themselves, we have 



fc,i = 1 + 6 [b(Tm - 1) - c(m - 1)] 
fo,i = 1 + 5 [brrii} . 



Therefore the fitness of cooperators at position i becomes 



(Bl) 



"'• ■ = 1 + jf ( 6(m * " l) ~ c(n * ~ 1} _ E m ^ - !) | + °(^ 2 ) 



N 



(B2) 



which then leads to the expected change of cooperator frequency 

6 



(Ap) = 



N 2 



HE 1 



-(b-c)(J2; 



\ i 

b- 



iii in, ) - ( mjmjrij 



N 



- E 



Two new correlation types in the neutral stationary state appear 



miirij 



N Pr(S k = 1) 



N 



mirrij 



N 



1,3 



0(S 2 ). 



N 2 Pr(S k = S q = l) 



N 2 



(B3) 



(B4) 



This then leads to the general expression analogous to ( 50 1 for the threshold ratio 



(N -2)(z -h) + z- g 
(N-2)(g-h)-z + g 



The smallest valid population size is N = 3. In the N — > oo the threshold b/c ratio with self interaction 
without it (B5l are the same (51 1 in the leading order, and their difference is only of order l/N. 



(B5) 
(l50) and 



APPENDIX C: FROM AVERAGES TO CORRELATIONS 



Here we obtain the identities listed in (46 1. The variables mi and rij are fixed in any given state. Let us use the 



indicator function 1, which is 1(A) — 1 if event A is true and 1(A) = if event A is false. Of course the stationary 
average of the indicator function is the stationary probability of an event 



(l(A))=Pr(A) 



(CI) 



16 



and by 1(A, B) we mean 1(^4 n B) = 1(A) 1(B). Now in any given state we can express rij and rrii by the indicator 
functions 



k 

£l(X, = i)l(S, = l). 



(C2) 



The sum in ( 46a I becomes 



^ mirm = ^ 



k.q 



l(S k = 1)1(5, = 1) J] - i)l(X q 



MS k = S q = l)l(X k = X,) (C3) 

A' . q 



since the sum over i is simply 

^ l(X fc - i)l(X q =i) = J2 ^ Xk =i, X g = i) = l(X k = X,). 



(C4) 



Now taking the average of ( C3 1 in the stationary state we obtain 



E 



m\\ = Y,(l(S k = S q = l,X k = X q )) =J2^(Sk = S q = 1, X h = X q ), 

I o k,q k,q 



(C5) 



where we have used identity (CI I. Since all individuals are equivalent in the stationary state, the above probabilities 



are the same for any pair of individuals, hence from now on we consider k and q as two randomly chosen individuals, 
and write 



N 2 Pr(S k = S q = l, X k =X q ). 



(C6) 



The expression ( 46b I can be derived similarly, since 



E m i n i = E 



i(s, = i)^i(x i = »)i(A: J = i) 



= J2Hs q = i)i(x k = x q ) 

k j g 



and taking the average of ( C7 1 in the stationary state leads to 



^m iril | = ^Pr(S g = l,X k = X q ) = N 2 Pt(S q = l,X k = X q ) 



For the last expression ( 46c I we have 



E"' = E 



k. q .l 



5^i(s fc = i,x fc =j)i(x,=j) 



= J]l(S| = l) l(5 fc = l, X fe = X 9 ) 



which in the stationary state becomes 



= ]T Pr(S, = 5 fc = 1, X k = X q ) = N 3 Pr(S, = S k = 1, X k = X q ) 



k,q,l 



(C7) 



(C8) 



(C9) 



(CIO) 
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APPENDIX D: MORAN DYNAMICS 



In the Moran model we chose a random individual to die, and another (with replacement) to multiply with prob- 
ability proportional to the player's payoff. The newborn then replaces the dead individual. Otherwise the dynamics 
is the same as in the W-F case. The behavior of the Moran model is also very similar to the W-F model, and the 
results can be written in an identical form in the N — > oo limit, by defining the appropriate variables. 

We consider the neutral case of the Moran model first. Let us obtain the probability Pr(T = t) that the time to the 
most recent common ancestor (MRCA) of two randomly chosen individual is T — t. Let us calculate the probability 
Pqa that they had a common ancestor one update step before. It could happen only if the parent and the dying 
individuals were different, which happens with probability 1 — 1/iV. Then our two individuals have a common ancestor 
if one of them is the parent and the other is the newborn daughter, which has a probability 2 W—i ■ Hence having 
a common ancestor in the previous update step is 

/ 1 \ 11 2 , v 



Consequently the probability that the MRCA is exactly time T = t backward is 

Pr(T = t) = (1 - Pca^Pca = (l 4 V 1 4 CM) 



N 2 N 2 



If we introduce a rescaled time r = t/(N 2 /2), then in the N — > oo limit the coalescent time distribution (D2 ) converges 
to the same density function ^ as we obtained for the W-F model. 

Since in our model mutations (in strategies) and motion only happen at birth events, let us investigate the statistics 
of birth events in the Moran model. As we follow the ancestral lines of two randomly chosen individuals backward 
in time, we can obtain the probability Pb that a birth event happens in one update step, but the ancestral lines do 
not coalesce. In other words, Pb is the probability that at a given time one of the two individuals is the daughter 
but the other is not the parent. If the parent dies during this update step (which happens with probability 1/iV) 
one individual is the daughter with probability 2/N (and the other individual cannot be the parent). If the parent 
does not die (which happens with probability 1 — 1 /N) one of the individuals is the daughter and the other is not the 
parent with probability 2/N ■ (N — 2)/(N — 1). Hence the probability that there is a birth event in the ancestry of 
either individual during one elementary time step is 

f l\ 2 7V-2 12 2(N-1) , . 

p ° = { 1 -n)-n-n^T + n-n = ± n*- 1 (D3) 

In the continuous time limit with r = t/{N 2 /2), a birth event happens at rate N . Consequently a mutation happens 
at rate /i = Nu on the ancestral line of two individuals. Similarly, one of the two individual hops at rate v = Nv 
in each direction. In other words the distance between the two individuals changes at rate v in each direction. This 
means that the continuous time [N — > oo) descriptions of the Moran and the W-F models are the same, but N must 



be used for the Moran and 2N for the W-F model in the definition of /i and v. Hence all N — > oo results of Section III 
are also valid for the Moran model. (Note that the diffusion coefficient of the cluster is D = v/N.) 

All formulas of Section |IV| are almost identical to those for W-F model. The average frequency of cooperators 



depends on the change of cooperators very similarly to (34 1 



1 l — ii 

(p) = ^ + N <A P ) scl (D4) 

2 u 

Instead of the fitness of the W-F model ( |30| , we have a very similar expression for the fitness after one elementary 
step 

N-l fa 

Wc >* = at + v^l i rf w — T ( D5 ) 



where the payoffs are again given by (29 1. Here the first term corresponds to the cooperator staying alive, and to 



second to it being chosen for reproduction. In the 5 — > limit (D5l becomes 




wc,i = 1 + ^| brrii - cn t — - mjUj | + 0(5 2 ) ! DO ! 



3 



Note that this is exactly the fitness of the W-F process (32 1 with a scaled selection strength 5' = S/N. Hence all 



results of Section IV and in particular the citical b/c ratio (52 1 are also valid for the Moran model 
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APPENDIX E: GENERAL PAYOFF MATRIX 



Instead of the payoff matrix (28) of the simplified Prisoner's Dilemma (PD) game, we study now a general payoff 
matrix 



R S' 
. T P, 



A similar derivation to the one presented in Section IV leads to the condition for cooperation 



(R - S)g + (S - P)z > (R- S -T + P)rj +{S + T - 2P)h 



(El) 



(E2) 



in the N — > oo limit, which is the analogous formula to (51 1. Here a new type of three point correlation must be 
introduced 



r, = Ft(S l = S k = S q , X k = X q ) 
In the v — > oo and fj, — > limit the correlations are 



1 



1 



h = 



2^ 
1 (, 2 



7] = 



2^ 
1 

2jv 



(E3) 



(E4) 



1-H- 



up to 0(1/ v) and 0(/i 2 ) terms. Here z, g, and h were obtained as limits of the general expressions (121, (21 1, and (26l 



respectively. The value of r/ was derived analogously to ( 25 ) . By substituting these correlations into ( E2 1 we finally 
arrive at the general condition for cooperation 



T-S < (R-P)(l + y/3) 



(E5) 



For the simplified PD game ( 28 1 we recover fl54| in the leading order 



For a non-degenerate payoff matrix, with the exchange of players R > P can always be achieved. Then under weak 
selection one can define an equivalent matrix 



with only two parameters 



1 a 
1 + 13 0, 



S-P n T-R 
a = — , f) = 



R-P 



R-P 



(E6) 



(E7) 



In these variables the condition for cooperation (E5l becomes 

< a + V3 (E8) 

which describes a straight threshold line in the (a, (3) plane (see Figure |5|. 

In Figure [5] we show how this threshold line (E8l divides the (a,/3) plane into a cooperative and a defective half 
plane. Three regions, bounded by black lines, correspond to the "Snow drift", the "Stag hunt" and the "Prisoner's 
dilemma" games. The blue straight lines on the (a, /?) plane correspond to the following representative simplified 
payoff matrixes 



Snow drift 
Stag hunt 
Prisoner's dilemma 



b-c/2 b~c 
b , 

b — c — c ' 


b — c —c ' 
b 



/3 = 1- a, with < a < 1 
(3 = -1, with a < 
(3 = —a, with a < 



(E9) 



Form the general condition (E5| we can deduce the condition for cooperation for these simplified games. There is 



always cooperation in the simplified Snow drift game. Cooperation is favored in the simplified Stag hunt game only 
for b/c> 1 + 1/(1 + V3). In the simplified PD game cooperators win for b/c > 1 + 2/^3 in agreement with (|54|. 
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FIG. 5: "Snow drift", "Stag hunt" and "Prisoner's dilemma" games correspond to three distinct regions in the (a,/3) plane, 
bounded by black lines. The red (thick) line (E8l marks the boundary between defection (yellow-shaded) and cooperation 
(white). The blue (thicker dashed) lines depict the corresponding simplified payoff matrices. 



APPENDIX F: RANDOMLY CHANGING PHENOTYPES 

Here we replace the one-dimensional phcnotype space with an infinite-dimensional phenotype space. We do not 
model the number of dimensions explicitly, but simply assume that every mutation causes a jump to a new unique 
phcnotype. Now the only way that two individuals can have the same phenotype is if there are no phenotypic 
mutations in their ancestry back to the time of their most recent common ancestor. This property is called identity by 
descent in population genetics and this mutation model known as the infinitcly-many-alleles, or simply infinite-alleles, 
mutation model [5cJl[57] . 

Let v be the probability that the phenotype of an offspring differs from that of its parent. Note that in the one- 
dimensional model, there is a mutation probability of v in each direction. As before, in the limiting (N — > oo) model 
with time rescaled appropriately, the phenotypic mutation rate to two individuals is equal to v. In the Wright-Fisher 
model we have 2Nv — > v (and Nv — > v in the Moran model), where the arrows correspond to the limit N — > oo. The 
definition of /i = 2Nu in the Wright-Fisher model (/i = Nu in the Moran model) is the same as before. 

Given a coalescence time r between a pair of individuals, 



C(r) 



(Fl) 



is the probability that they have the same phenotype. Therefore, in the N — > oo limit, the correlations defined in ^ 
become 



1 f 1 



h = 



1 

2^1 + 1^ ' l + (j, + i> 

1 1/1 1 1 

; 1- o 1- ; H « 

l + v S + n + vyi + v l + n 1 + fi + iy 



(F2) 



The calculation goes analogously to that of Section III The threshold parameters (51 1 for cooperation to be favored 
becomes 



y(3 + 2/x + t/) + (l + /x)(3 + /j) 
i/(2 + n + v) 



(F3) 
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FIG. 6: Exact threshold b/c ratio ( F3 1 for randomly changing phenotypes for TV — > oo. Cooperation is most favored in the 
v — > oo limit, where (b/c)* — 1. Note that the lines for finite values of v are not straight. 



This is plotted in Figure [6] which can be compared to the corresponding Figure [4] for the one-dimensional model. 

Cooperation is most favored when v is large because in this case two individuals that share the same phenotype 
will almost surely have the same strategy. We have 



= 1 



1 + H 



+ 0(v- 2 ) 



(F4) 



In the v — > oo limit, (b/c)* — 1, i.e. cooperation is favored whenever the benefit b from cooperation is larger than the 
cost c. 



For general payoff matrices (Ell, we restrict our calculation to the fi — » limit. The calculation is completely 
analogous to that of Appendix EjFirst we calculate the three point correlation rj, which is defined in (E3). Up to 
first order in [i we obtain 



1 



l + v 



9 + Iv + 2v 2 
4(1 + i/)(3 + i/) 



(F5) 



Substituting this expression together with (F2) into the general condition (E2) for cooperation, we finally obtain 

(1 + i/)(3 + 2i/) 



T-S <{R-P) 



(F6) 



This result is valid for general values of v. For v — > condition (F6) becomes T — S < R — P, while in the v — > oo 
limit it is simply R > P. 



By using the scaled variables a, (3, introduced in (E6), condition (F6) is again a straight line in the (a,/?) plane. 
For v — > there is no cooperation in the PD region (see this region in Figure [5| , but for v — > oo the whole plane 
corresponds to cooperation. 
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